Towards a portable and future-proof particle-in-cell plasma physics code

We present the first reported OpenCL implementation of EPOCH3D, an extensible particle-in-cell plasma physics code developed at the University of Warwick. We document the challenges and successes of this porting effort, and compare the performance of our implementation executing on a wide variety of hardware from multiple vendors. The focus of our work is on understanding the suitability of existing algorithms for future accelerator-based architectures, and identifying the changes necessary to achieve performance portability for particle-in-cell plasma physics codes. We achieve good levels of performance with limited changes to the algorithmic behaviour of the code. However, our results suggest that a fundamental change to EPOCH3D’s current accumulation step (and its dependency on atomic operations) is necessary in order to fully utilise the massive levels of parallelism supported by emerging parallel architectures

Performance modelling and optimisation of inertial confinement fusion simulation codes

Legacy code performance has failed to keep up with that of modern hardware. Many new hardware features remain under-utilised, with the majority of code bases still unable to make use of accelerated or heterogeneous architectures. Code maintainers now accept that they can no longer rely solely on hardware improvements to drive code performance, and that changes at the software engineering level need to be made. The principal focus of the work presented in this thesis is an analysis of the changes legacy Inertial Confinement Fusion (ICF) codes need to make in order to efficiently use current and future parallel architectures. We discuss the process of developing a performance model, and demonstrate the ability of such a model to make accurate predictions about code performance for code variants on a range of architectures. We build on the knowledge gained from such a process, and examine how Particle-in-Cell (PIC) codes must change in order to move towards the required levels of portable and future-proof performance needed to leverage the capabilities of modern hardware. As part of this investigation, we present an OpenCL port of the legacy code EPOCH, as well as a fully featured mini-app representing EPOCH. Finally, as a direct consequence of these investigations, we directly apply these performance optimisations to the production version EPOCH, culminating in a speedup of over 2x for the core algorith

Performance optimisation of inertial confinement fusion codes using mini-applications

Despite the recent successes of nuclear energy researchers, the scientific community still remains some distance from being able to create controlled, self-sustaining fusion reactions. Inertial Confinement Fusion (ICF) techniques represent one possible option to surpass this barrier, with scientific simulation playing a leading role in guiding and supporting their development. The simulation of such techniques allows for safe and efficient investigation of laser design and pulse shaping, as well as providing insight into the reaction as a whole. The research presented here focuses on the simulation code EPOCH, a fully relativistic particle-in-cell plasma physics code concerned with faithfully recreating laser-plasma interactions at scale. A significant challenge in developing large codes like EPOCH is maintaining effective scientific delivery on successive generations of high-performance computing architecture. To support this process, we adopt the use of mini-applications -- small code proxies that encapsulate important computational properties of their larger parent counterparts. Through the development of a mini-application for EPOCH (called miniEPOCH), we investigate a variety of the performance features exhibited in EPOCH, expose opportunities for optimisation and increased scientific capability, and offer our conclusions to guide future changes to similar ICF codes

Mini-app driven optimisation of inertial confinement fusion codes

In September 2013, the large laser-based inertial confinement fusion device housed in the National Ignition Facility at Lawrence Livermore National Laboratory, was widely acclaimed to have achieved a milestone in controlled fusion – successfully initiating a reaction that resulted in the release of more energy than the fuel absorbed. Despite this success, we remain some distance from being able to create controlled, self-sustaining fusion reactions. Inertial Confinement Fusion (ICF) represents one leading design for the generation of energy by nuclear fusion. Since the 1950s, ICF has been supported by computing simulations, providing the mathematical foundations for pulse shaping, lasers, and material shells needed to ensure effective and efficient implosion. The research presented here focuses on one such simulation code, EPOCH, a fully relativistic particle-in-cell plasma physics code, developed by a leading network of over 30 UK researchers. A significant challenge in developing large codes like EPOCH is maintaining effective scientific delivery on successive generations of high-performance computing architecture. To support this process, we adopt the use of mini-applications – small code proxies that encapsulate important computational properties of their larger parent counterparts. Through the development of a miniapp for EPOCH (called miniEPOCH), we investigate known timestep scaling issues within EPOCH and explore possible optimisations: (i) Employing loop fission to increase levels of vectorisation; (ii) Enforcing particle ordering to allow the exploitation of domain specific knowledge and, (iii) Changing underlying data storage to improve memory locality. When applied to EPOCH, these improvements represent a 2.02× speed-up in the core algorithm and a 1.55× speed-up to the overall application runtime, when executed on EPCC’s Cray XC30 ARCHER platform

Cusp energetic particle events: Implications for a major acceleration region of the magnetosphere

The Charge and Mass Magnetospheric Ion Composition Experiment (CAMMICE) on board the Polar spacecraft observed 75 energetic particle events in 1996 while the satellite was at apogee. All of these events were associated with a decrease in the magnitude of the local magnetic field measured by the Magnetic Field Experiment (MFE) on Polar. These new events showed several unusual features: (1) They were detected in the dayside polar cusp near the apogee of Polar with about 79% of the total events in the afternoonside and 21% in the morningside; (2) an individual event could last for hours; (3) the measured helium ion had energies up to and many times in excess of 2.4 MeV; (4) the intensity of 1–200 KeV/e helium was anticorrelated with the magnitude of the local geomagnetic field but correlated with the turbulent magnetic energy density; (5) the events were associated with an enhancement of the low-frequency magnetic noise, the spectrum of which typically extends from a few hertz to a few hundreds of hertz as measured by the Plasma Wave Instrument (PWI) on Polar; and (6) a seasonal variation was found for the occurrence rate of the events with a maximum in September. These characterized a new phenomenon which we are calling cusp energetic particle (CEP) events. The observed high charge state of helium and oxygen ions in the CEP events indicates a solar source for these particles. Furthermore, the measured 0.52–1.15 MeV helium flux was proportional to the difference between the maximum and the minimum magnetic field in the event. A possible explanation is that the energetic helium ions are energized from lower energy helium by a local acceleration mechanism associated with the high-altitude dayside cusp. These observations represent a potential discovery of a major acceleration region of the magnetosphere

Comparing verbal autopsy cause of death findings as determined by physician coding and probabilistic modelling: a public health analysis of 54 000 deaths in Africa and Asia.

BACKGROUND: Coverage of civil registration and vital statistics varies globally, with most deaths in Africa and Asia remaining either unregistered or registered without cause of death. One important constraint has been a lack of fit-for-purpose tools for registering deaths and assigning causes in situations where no doctor is involved. Verbal autopsy (interviewing care-givers and witnesses to deaths and interpreting their information into causes of death) is the only available solution. Automated interpretation of verbal autopsy data into cause of death information is essential for rapid, consistent and affordable processing. METHODS: Verbal autopsy archives covering 54 182 deaths from five African and Asian countries were sourced on the basis of their geographical, epidemiological and methodological diversity, with existing physician-coded causes of death attributed. These data were unified into the WHO 2012 verbal autopsy standard format, and processed using the InterVA-4 model. Cause-specific mortality fractions from InterVA-4 and physician codes were calculated for each of 60 WHO 2012 cause categories, by age group, sex and source. Results from the two approaches were assessed for concordance and ratios of fractions by cause category. As an alternative metric, the Wilcoxon matched-pairs signed ranks test with two one-sided tests for stochastic equivalence was used. FINDINGS: The overall concordance correlation coefficient between InterVA-4 and physician codes was 0.83 (95% CI 0.75 to 0.91) and this increased to 0.97 (95% CI 0.96 to 0.99) when HIV/AIDS and pulmonary TB deaths were combined into a single category. Over half (53%) of the cause category ratios between InterVA-4 and physician codes by source were not significantly different from unity at the 99% level, increasing to 62% by age group. Wilcoxon tests for stochastic equivalence also demonstrated equivalence. CONCLUSIONS: These findings show strong concordance between InterVA-4 and physician-coded findings over this large and diverse data set. Although these analyses cannot prove that either approach constitutes absolute truth, there was high public health equivalence between the findings. Given the urgent need for adequate cause of death data from settings where deaths currently pass unregistered, and since the WHO 2012 verbal autopsy standard and InterVA-4 tools represent relatively simple, cheap and available methods for determining cause of death on a large scale, they should be used as current tools of choice to fill gaps in cause of death data